Hadoop环境下基于SparkSQL海量自动站数据查询统计初探     

黄志  詹利群  任晓炜  李涛 《气象科技》2019,47(5):768-772

在Hadoop分布式计算和存储架构下,自定义ETL数据清洗规则将海量自动站小时单站文件按所属年和站号合并为大文件流转存储至HDFS中,并运用SparkSQL并行计算框架进行统计处理生成常用气象要素日统计值。结果表明,数据处理和获取时效较关系型数据库方式有显著提升。采用SparkSQL并行计算框架对多气象要素多站点和长时间序列进行数据统计处理查询均能达到秒级别响应,并随着统计站点数的不断增加和时间跨度的延长其优势更为明显,能更高效地支撑此类气象数据服务,为海量气象数据处理从关系型数据库到大数据分布式架构的转换处理提供了新思路。  相似文献   

地面气象观测无人值守系统的设计与应用          下载免费PDF全文

任晓炜  马军  李静锋  邓力涌  廖伟平  张小琼 《气象与环境科学》2019,42(3):129-135

气象观测数据的传输率和可用率是气象预报服务及时准确的基础,是衡量通信系统运行可靠程度及运维保障能力的关键指标。长期以来,各级业务人员通过不间断值守班来保障气象观测数据的传输质量。随着现代化气象业务体系的不断发展,气象业务对资料传输的时效性、完整性要求也越来越高,对气象资料的综合监控能力也提出了更高的要求。在全国地面气象业务体制改革的要求下,广西率先在全国实行了基层台站无人值守业务调整,设计并实现广西地面气象观测无人值守系统。该系统集数据传输监控、数据传输时效统计于一体,通过自动监控告警、主备站资料(要素)智能识别替代、双条传输链路热备等设计,建立一套无人值守传输业务流程,从技术上支撑业务改革调整,大大降低基层业务人员工作强度。业务应用分析表明,无人值守系统运行可靠,业务质量保持高水平稳定,业务效益凸显。  相似文献   

Deep electrical structure and its geodynamic significance beneath the Middle and Lower Reaches of the Yangtze River metallogenic belt     

Xianying Wang  Lincheng Zhang  Xiao Xiao  Cong Zhou  Zhengyong Ren  Jingtian Tang 《《地质学报》英文版》2018,92(Z1):335-336

  相似文献   

Generation and Development of Structures and Their Controls on Ore Mineralization in the Lehong Zinc-lead Deposit,Northeastern YunnanEI北大核心CSCD     

Cui J.  Han R.  Wang J.  Ren T.  Wu Y.  Zhao D. 《大地构造与成矿学》2018,(4):664-680

The Lehong zinc-lead deposit is one of the new-found large Zn-Pb deposits in the Sichuan- Yunnan-Guizhou Zn-Pb poly-metallic mineralization area. The ore-bodies of the Lehong deposit are strictly controlled by structures, and the fault tectonites can be divided into four groups which include morbruk rock, cataclastic rock and tectonic breccia. The mechanical properties of the faults suggest that there are five tectonic systems formed in the Caledonian-early Hercynian, Indosinian-early Yanshanian, mid-Yanshanian, late Yanshanian and Himalayan periods, respectively. The Lehong fault and the Qiaojia-Lianfeng fault are two important ore-controlling structures which might have acted as channel ways of the ore fluids. The ores are hosted in the subordinary fracture zone, joint fracture zone and interlayer fault zones in trailing folds of the Lehong fault and the Baobaoshang syncline and Jinjiagou anticline. Hence, the Lehong deposit is a tectonic -controlled ore deposit. These research do not only lay a foundation for studying deposit genesis and ore-finding direction in the Lehong deposit, but also provides important enlightenments for the ore-finding prognosis of lead-zinc deposits in northwestern Guizhou ore concentration district. © 2018, Science Press. All right reserved.  相似文献   

Geochemistry and geochronology of Late Jurassic and Early Cretaceous intrusions related to some Au (Sb) deposits in southern Anhui: a case study and review     

Qing Hu  Huangling Gu  Xiaoyong Yang  Yisu Ren  Ergen Gao  Zhangxing Nie 《中国地球化学学报》2018,37(3):360-383

Some Au deposits in southern Anhui Province have recently been found to be closely associated with Late Mesozoic intrusions. Typical examples include the Huashan Au (Sb) deposit and Au deposits at Zhaojialing, Wuxi, and Liaojia. In order to understand the mechanisms that led the formation of these Au deposits, we make detailed reviews on the geological characteristics of these Au deposits. Specifically, we present new LA-ICP-MS zircon U–Pb dating, along with elemental and Hf isotopic data from the Huashan Au (Sb) deposit. Our data suggests that the Huashan ore-related intrusions were emplaced during the Late Jurassic and Early Cretaceous periods (144–148 Ma). They are characterized by arc-magma features and high oxygen fugacity and are rich in inherited zircons. Zircon U–Pb ages and Lu–Hf isotopes from intrusions suggest that Proterozoic juvenile lithosphere is the main source of these intrusions. The regional geological history implies that lithosphere beneath southern Anhui was produced during a Proterozoic subduction and was fertilized with Au (Cu) in the process. Integrated with the results of previous studies, we inferred that Late Mesozoic intrusions formed by the remelting of the lithosphere could provide the metal endowment for the Au-rich deposits in southern Anhui.  相似文献   

Late Quaternary Slip Rate of the Xiugou Segment,Eastern Kunlun Fault Zone   总被引：1，自引：0，他引：1  

Feipeng Huang  Junjie Ren  Yanwu Lü  Junxiang Zhao 《地球科学进展》2018,33(3):321-332

The Eastern Kunlun fault zone (EKLF) is a large left-lateral strike-slip fault, whose slip rate is meaningful to seismic hazard assessment and geodynamics of the Tibetan Plateau. Previous studies suggested that the late Quaternary average slip rate was stable and uniform (10~13 mm/a) in the central and western segment of the EKLF. But there were a few researches of accurate slip rate in the central segment on the EKLF. Therefore, we focused on an offset and well preserved alluvial fan from Xiugou basin, located in the east of Xidatan-Dongdatan, to make it clear. Moreover, we used high-resolution satellite images and digital elevation model extracted from SPOT7 stereo image pairs to restore the offset alluvial fan, and combined terrestrial cosmogenic nuclides method, including 13 quartz-rich samples from this fan surface, 1 quartz-rich sample from the main active channel bed and 1 ¹⁰Be depth profile from this fan edge to eliminate the ¹⁰Be concentration of inheritance accurately, with 1 optically stimulated luminescence sample to obtain the reliable age of this alluvial fan together. Referring to field observations, this alluvial fan was offset left-laterally by (1 862±103) m, and its age is (76.55±3.20)~(106.37±3.38) ka which can be determined through the actual geologic setting and improving chi-square test. Thus, we used the Monte Carlo method to obtain a left-lateral slip rate of (20.3+3.5/-2.3) mm/a with 68% confidence envelopes since the late Pleistocene in the Xiugou basin. As a result, combining with the results of previous studies, the left-lateral slip rate indicated that the obviously decreasing activity transferred from late Pleistocene to Holocene on the central segment of the EKLF.  相似文献   

基于GIPL2模型的青藏高原活动层土壤热状况模拟研究   总被引：5，自引：5，他引：0  

秦艳慧  吴通华  李韧  胡国杰  乔永平  朱小凡  杨淑华  余文君  王蔚华 《冰川冻土》2018,40(6):1153-1166

青藏高原活动层土壤热状况,对深入了解高原活动层的厚度变化特征、下垫面的热力作用以及对气候变化预测均有重要意义。利用GIPL2模型模拟青藏高原多年冻土区不同植被状况下活动层土壤热状况。模拟结果表明：模型在高寒草原（QT06）试验点模拟效果较好,高寒沼泽草甸（QT03）试验点的模拟效果较差,高寒草甸（QT01）、高寒荒漠草原（QT05）和高寒草原化草甸（QT04）试验点的模拟效果介于高寒草原试验点和高寒沼泽草甸试验点之间。QT01、QT03、QT04、QT05和QT06的土壤温度模拟值与观测值相比,均方根误差分别为0.67、1.29、0.73、0.7和0.56℃;相关系数分别为0.99、0.87、0.98、0.98和0.96;平均误差分别为0.37、0.61、0.31、0.45和0.16℃。QT06模拟结果较好,原因在于此点土壤质地变化不大,模型的分层与所取的参数更加接近此点的实际状况。QT03模拟结果较差,可能由于此地区土壤中存在砾石,在导热率参数化方案中没有考虑砾石含量,导致模拟结果偏差较大。总体而言,GIPL2模型对青藏高原活动层土壤热状况的模拟具有一定的优势,是一种模拟多年冻土区活动层土壤热状况较为理想的模型。  相似文献   

准噶尔盆地腹部地区生烃增压定量化评价——东道海子北凹陷为例          下载免费PDF全文

于春勇郭小文  任战利任文波 杨桂林 《地质科学》2018,(1):207-220

目前准噶尔盆地腹部地区超压演化过程仅提供了相对时间、概念、定性的认识,并未达到定量化。本文以东道海子北凹陷为例,基于流体包裹体、单井一维和剖面二维烃源岩成熟生烃史模拟技术,采用生烃增压模型定量化评价了侏罗纪八道湾组顶部和底部烃源岩的生烃增压演化过程。研究结果表明侏罗纪八道湾组为东道海子北凹陷主力烃源岩,具有幕式排烃的特点。八道湾组底部烃源岩经历了3 次超压增加和两次超压释放的过程,顶部烃源岩经历了两次超压增加和一次超压释放的过程。八道湾底部烃源岩3 次超压增加的时间分别为距今170～70 Ma、55～23 Ma、10～0 Ma,对应最大压力系数均为2.0,最大剩余流体压力50 MPa,两次超压释放的时间分别为70～55 Ma、23～10 Ma;八道湾组顶部烃源岩两次超压增加的时间分别为130～40 Ma、23～0 Ma,对应流体最大压力系数最大亦可达2.0 左右,最大剩余流体压力44 MPa,超压释放的时间为40～23 Ma。  相似文献   

断裂应力特征及其转折控油气规律研究——以南堡凹陷为例     

马强  德勒恰提&#;加娜塔依  陈亚军  马乐  于景维  祁利祺 《地质力学学报》2018,24(5):587-597

为了分析断裂对油气宏观分布规律的控制作用,寻找有勘探潜力的油气富集带,文中以地质力学理论为指导,对南堡凹陷断裂组合应力特征、断裂转折及成因进行解析,并结合砂体分布和古今构造应力场分布特征,对南堡凹陷东营组油气分布规律和有利勘探区带做了研究。结果表明:断裂转折部位对油气分布有着控制作用,东营组已探明的油气主要在帚状断裂系、入字形断裂系、交织式断裂系转折轴部高曲率附近呈环带状分布;断裂转折凹面一侧控油性明显,从宏观上来看,油气具有沿沉积相带由细到粗的构造脊高点以及弧形断裂转折凹面一侧的选择性运移的特征,断裂转折轴部高曲率高孔、高渗域与高孔、高渗砂体沉积相带的有利对置决定了油气的主流向;综合研究指出,南堡凹陷北堡构造带是今后优势勘探的首要区带,其次是南堡构造带和南堡4号蛤坨构造带南部。  相似文献   

克拉玛依地区野外地质实习基地建设研究     

孙新铭  李莉  于景维  徐媛媛  柳妮  王满  井春丽 《城市地质》2018,(2):109-114

野外实践教学在地学人才培养中具有重要作用,稳定的实习基地是开展实践教学的前提和基础。克拉玛依所处的准噶尔盆地西北缘地区位于哈萨克斯坦、塔里木与西伯利亚三大板块的交汇处,自晚古生代以来经历了多期次、多阶段长期而复杂的地质演化,褶皱构造、大型纵向走滑断裂构造发育,广泛发育晚古生代花岗岩侵入体和蛇绿混杂岩带,沉积地层连续、完整,沉积构造典型、特征显著,古生物化石丰富,白杨河深切峡谷、河流阶地地貌以及乌尔禾魔鬼城雅丹风蚀地貌各具特色,西北缘油砂矿、紫砂矿、金丝玉等矿藏储量丰富。区域内的地层、古生物、构造、地貌、矿产资源等为克拉玛依工程教育基地各院校的地学类相关专业的野外实践教学提供了丰富的教学资源,是一个理想的野外地质实习基地。  相似文献